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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE _3_ MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
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earned patent term adjustment. See 37 CFR 1.704(b). 
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Responsive to communication(s) filed on J^T January 2004 . 
2a)D This action is FINAL. 2b)K This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 1 1 , 453 O.G. 213. 
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4) ^ Claim(s) 122-170 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claimfe) 122-164. 166, 168 and 170 is/are rejected. 

7) ^ Claim(s) 165.167 and 169 is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 
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10) 13 The drawing(s) filed on 23 August 2001 is/are: a)^ accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
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DETAILED ACTION 
Continued Examination Under 37 CFR LI 14 

1 . A request for continued examination under 37 CFR 1. 1 14, including the fee set forth in 37 
CFR 1 . 17(e), was filed in this application after final rejection. Since this application is eligible for 
continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 1.17(e) has been 
timely paid, the finality of the previous Office action has been withdrawn pursuant to 

37 CFR 1.114. 

The Applicant's PRELIMINARY AMENDMENT AND PETITION FOR EXTENSION 
OF TIME, filed on January 14, 2004 (paper 16), has been entered. An action continuing 
examination on the merits follows. The text of those sections of Title 35, U.S. Code not included 
in this action can be found in a prior Office action. 

Information Disclosure Statement 

2. The information disclosure statement filed January 14, 2004 (paper 1 5) fails to comply 
with 37 CFR 1.98(a)(1), which requires a list of all patents, publications, or other information 
submitted for consideration by the Office. Because its defects appear to be listing errors, as 
inaccuracy in recording the author, title, date, or pages, the Examiner has changed citations on the 
disclosure statement to correspond to the copies provided. If the changes and/or additions are 
acceptable to the Applicant, no action is required. Further submissions should comply with 37 
CFR 1.97 and 37 CFR 1.98 as of the date of their submission. 
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Specification 

3. The specification is objected to using the same rationale as in the previous Office actions 
(paper 14 and paper 10) as failing to provide proper antecedent basis for the following claimed 
subject matter: data arranged in blocks of equal time duration (at least claim 128). Appropriate 
correction is required. 

Claim Informalities 

4. Claims 165, 167, and 169 are objected to as being (directly or indirectly) dependent upon a 
rejected base claim. See MPEP § 608.01(n)V. The claim(s) would be allowable over the prior art 
of record if rewritten to include all of the limitations of the base claim and any intervening claims. 

5. Claim 146 is objected to under 37 CFR 1.75(a) because the meaning of the phrase "each 
block's location" needs clarification. The language is such as to suggest that a block was 
previously recited in the claim or in claims from which it depends. To further timely prosecution 
and evaluate prior art, the Examiner has interpreted this phase to refer to -a block's location--. 

6. Regarding claim 163, the phrase "the audio data" (lines 3-4) lacks antecedent basis in the 
claim. To further timely prosecution and evaluate prior art, the Examiner has interpreted this 
phase to refer to -the voice signal--. 

7. Claim 169 is objected to as failing to define the invention with the clarity required by 37 
CFR 1.75(a). Because it is written in dependent form, claim 169 must be construed to incorporate 
by reference all limitations of claims 123, 166, and 167, to which it refers. Claims 123, 166, and 
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167 include an apparatus, and input, a speech recognizer, and an annotation generator; therefore, 
claim 169 must include those elements. However, as written, claim 169 could appear to be 
attempting to claim only a computer program code embodiment of its parent claims. The 
Applicant should cancel the claim or amend the claim to further limit the parent claim with the 
clarity required by 37 CFR 1.75(a). To clearly claim only a program code embodiment, the 
Applicant should rewrite the claim in independent form and as computer-executable code 
embodied on a computer-readable medium. 

8. The Applicant is advised that should claim 160 be found allowable, claim 161 will be 
objected to under 37 CFR 1.75 as being a substantial duplicate thereof When two claims in an 
application are duplicates or else are so close in content that they both cover the same thing, 
despite a slight difference in wording, it is proper after allowing one claim to object to the other as 
being a substantial duplicate of the allowed claim. See MPEP § 706.03(k). 

Claim Rejections - 35 USC § 112 

9. Claims 149, 150, 152, and 153 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

10. Regarding claim 149, the phrase "phonemes associated with said links" (line 2) lacks 
antecedent basis in the claim. No links were previously said and no phonemes were said to be 
associated with links. To further timely prosecution and evaluate prior art, the Examiner has 
interpreted this phase to refer to —phonemes--. 
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11. Claims 150, 152, and 153 are indefinite in the same way as claim 149 because the 
limitations are recited using obviously similar phrases. 

Claim Rejections - 35 USC §101 

12. Claim 168 is rejected under 35 U.S.C. 101 because the claimed invention is directed to 
non-statutory subject matter. 

The limitations of claim 168 are directed to computer program code of instructions. A list 
of computer code as instructions comprises only functional descriptive material. The description 
of instructions is not a physical article, nor is it a statutory process, as it is not acts being 
performed. Instructions that are not encoded on a computer-readable medium do not define both 
functional and structural interrelationships between the instructions and the medium, which permit 
the instructions' functionality to be realized. Functional descriptive material that is not claimed as 
embodied in computer-readable media is descriptive material per se and is not statutory because it 
is neither a useful process, machine, manufacture, nor composition. Functional descriptive 
material on computer-readable media carries out an algorithm that electrically changes a general 
purpose computer into a special purpose machine by activating electrical paths and deactivating 
other paths. All claim limitations and have been considered, and the matter of claim 168 has been 
found nonstatutory as a computer program per se, because the underlying process describing the 
instructions does not provide them with the necessary functional and structural interrelationship 
with any medium to satisfy the requirements of 35 U.S.C. 101. 



r 
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Claim Rejections - 35 USC § 102 

13. The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed in 
the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

Meador 

14. Claims 122, 123, 131-133, 135-137, 141, 142, 148-150, 152-154, 168, and 170 are rejected 
under 35 U.S.C. 102(b) as being anticipated by Meador . m et al. [US Patent 5,638,425]. 

15. Regarding claim 122, Meador [at column 13, lines 4-39] describes an embodiment of 
phoneme data that annotates an audio file of utterances. Reference to the techniques of columns 
10-22 is by way of example of details that supplement the discussions of columns 1-9. Meador 
explicitly describes the following elements: 

audio data in a data file [at column 6, line 67, as the utterance of the city is stored in a file]; 

ASR generates phoneme data for that audio data [at column 1 1, lines 1-22, as a phoneme- 
recognizer analyzes from the point of view of phonemes the utterances for each city on a list]; 

a word decoder identifies words [at column 10, lines 43-45, as a word recognizer generates 
candidate cities]; 

the identified words are within the ASR-generated phoneme data [at column 1 1, lines 45- 
48, as the city candidates are recognized by the phoneme recognizer]; 

an annotation generator generates annotation data by combining the generated phoneme 
data and the decoder-identified words [at column 7, lines 10-30, as the process associates words 
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and phonemes by linking the phoneme strings using experimental utterances and the city- 
candidates from the word recognizer]. 

16. Regarding claim 123, Meador [at column 13, lines 4-39] describes an embodiment of 
phoneme data that annotates an audio file of utterances. Reference to the techniques of columns 
10-22 is by way of example of details that supplement the discussions of columns 1-9. Meador 
explicitly describes the following elements: 

an input receives an input voice signal [at column 8, lines 1-3, as VPU cards direct a 
spoken city name]; 

a speech recognizer converts it into phoneme data [at column 1 1, lines 1-22, as a phoneme- 
recognizer analyzes from the point of view of phonemes the utterances for each city on a list]; 

a speech recognizer converts it into words [at column 10, lines 43-45, as a word recognizer 
generates candidate cities]; 

an annotation generator generates annotation data by combining the generated phoneme 
data and the decoder-identified words [at column 7, lines 10-30, as the process associates words 
and phonemes by linking the phoneme strings using experimental utterances and the city- 
candidates from the word recognizer]. 

1 7. Regarding claim 131, Meador also describes: 

speech of a plurality of speakers [at column 1 1, lines 1-3, as utterances of each speaker of a 
set of speakers]; 

separate phoneme annotation data for speech of each speaker [at column 12, line 63- 
column 13, line 17, as retained phoneme strings from the utterances of all speakers]; 
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separate word annotation data for speech of each speaker [at column 12, lines 1-1 1, as the 
stored word candidates for the utterances of the set of speakers]. 

18. Regarding claim 132, Meador also describes: 

weighting for those phonemes [at column 13, lines 1-20, as the frequency/probability for 
each resulting string]. 

19. Regarding claim 133, Meador also describes: 

weighting for those identified words [at column 11, lines 1-8, as weights or proportions 
applied to the recognized results]. 

20. Claim 135 sets forth additional limitations similar to limitations set forth in claim 132. 
Meador describes the additional limitations as indicated there. 

21. Claim 136 sets forth additional limitations similar to limitations set forth in claim 133. 
Meador describes the additional limitations as indicated there. 

22. Regarding claim 137, Meador also describes: 

an associator associating the annotation data with the data file [at column 6, line 67- 
column 8, line 52, as word scores and confidence scores that the word and phonemes were in fact 
the one that the speaker was uttering and was stored in the file]. 
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23. Claim 141 sets forth a method with limitations comprising the functionality associated 
with using the system recited in claim 122. Because Meador describes the similar limitations as 
indicated there, this claim thus is anticipated accordingly. 

24. Claim 142 sets forth a method with limitations comprising the functionality associated 
with using the system recited in claim 123. Because Meador describes the similar limitations as 
indicated there, this claim thus is anticipated accordingly. 

25. Claim 148 sets forth a method with limitations comprising the functionality associated 
with using the system recited in claim 131. Because Meador describes the similar limitations as 
indicated there, this claim thus is anticipated accordingly. 

26. Claims 149, 150, 152, and 153 set forth a method with limitations comprising the 
functionality associated with using the system recited in claims 132, 133, 132, and 136, 
respectively, if the Examiner's assumptions that establish clarity for the limitations are correct, as 
described elsewhere in this Office action. Because Meador describes the similar limitations as 
indicated there, these claims thus are anticipated accordingly. 

27. Claim 154 sets forth a method with limitations comprising the functionality associated 
with using the system recited in claim 137. Because Meador describes the similar limitations as 
indicated there, this claim thus is anticipated accordingly. 
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28. Claim 168 sets forth limitations similar to claim 141. Meador describes the limitations as 
indicated there. Meador also describes additional limitations as follows: 

computer program code [at column 3, lines 32-35, as stored program for operation]. 

29. Claim 170 sets forth limitations similar to claim 141. Meador describes the limitations as 
indicated there. Meador also describes additional limitations as follows: 

a computer readable medium storing computer executable instructions [at column 3, 
lines 32-35, as a computer operating a stored program]. 



30. Claims 124, 138-140, 143, and 155-157 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Ahmad et al. [US Patent 6,172,675]. 

31. Regarding claim 124, Ahmad [at columns 5-6] describes an annotation embodiment 
recognizable as a whole to one versed in the art using text received by any appropriate manner by 
explicitly describing the content and functionality of the recited limitations as the following 
terminology: 

an input, a character recognizer, a converter, and an annotation generator [at column 22, 
lines 52-57, as methods implemented on a computer using appropriate programs for 
manipulating]; 

receives image data representing text [at column 5, lines 34-35, as a device to make 
available digitized data from analog data]; 



Ahmad 
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converts the image data into text [at column 5, lines 36-37, as optical character recognition 
processing]; 

converts words in the text into phoneme data [at column 15, lines 49-65, as a dictionary 
and rules transcribe the words in the text into phonemes]; 

generates annotation data by combining the phoneme data and the word data [at 
column 12, lines 15-28, as construct a network top tier of words and next lower tier of phonemes]. 

32. Regarding claim 138, Ahmad also describes: 

automatic phonetic transcription unit [at column 22, lines 52-57, as methods implemented 
on a computer using appropriate programs for manipulating]; 

generates phoneme data from words within the text data [at column 15, lines 49-65, as a 
dictionary and rules transcribe the words in the text into phonemes]; 

the text data was output [at column 7, lines 44-47, as the text data set is obtained]; 

the text data is by the character recognizer [at column 5, lines 36-37, as optical character 
recognition processing]. 

33. Regarding claim 139, Ahmad also describes: 

an associator [at column 22, lines 52-57, as methods implemented on a computer using 
appropriate programs for manipulating]; 

associates the annotation data with the text data (or its image) [at column 12, lines 50-51, 
as the network aligned with the text data]. 
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34. Regarding claim 140, Ahmad also describes: 

the input comprises a document scanner (or fax) [at column 5, lines 34-37, as an image 
scanner makes available digitized data from analog data]. 

35. Claims 143 and 155-157 set forth a method with limitations comprising the functionality 
associated with using the system recited in claims 124 and 138-140. Because Ahmad describes 
the similar limitations as indicated there, these claims thus are anticipated accordingly. 



Ahmad and Miike 

36. Claims 122, 123, 125-130, 134, 141, 144-147, 151, 158, 159, 162-164, 166 are rejected 
under 35 U.S.C. 103(a) as being unpatentable over Ahmad et al [US Patent 6,172,675] in view of 
Miike [US Patent 5,787,414]. 

37. Regarding claim 122, Ahmad and Miike make obvious the claimed limitations as a whole, 
because Ahmad [at columns 5-6] describes an annotation embodiment recognizable as a whole to 
one versed in the art using speech received by any appropriate manner and transcribed by any 
conventional automatic speech recognizer by explicitly describing the content and functionality of 
the recited limitations as the following terminology: 

an automatic speech recognizer, a word decoder, and an annotation generator [at 
column 22, lines 52-57, as methods implemented on a computer using appropriate programs for 
manipulating]; 



Claim Rejections - 35 USC § 103 
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generates data for audio in a data file [at column 7, lines 54-5 7, as produce a text data set 
by analysis using any know speech recognition]; 

generates annotation data by combining phoneme data and word data [at column 12, 
lines 15-28, as construct a network top tier of words and next lower tier of phonemes]. 

However, Ahmad does not describe details of speech recognition. In particular, Ahmad 
does not describe phoneme recognition and phonemes representing the words. 

The discussion by Ahmad requires a method for automatic speech recognition, but merely 
any conventional method. Miike provides a conventional method. Accordingly, an artisan would 
be motivated to find known automatic speech recognition programs in order to implement 
Ahmad 5 s method of audio/video/text time alignment. Ahmad 's [at abstract] application of search 
and access to databases would suggest finding speech recognition used in an similar application. 
Miike [at columns 52 ffj, then, discloses speech recognition for data retrieval, and describes: 

generates phoneme data for audio [at column 56, lines 25-27, as obtain the phoneme 
sequence of the acoustic data]; 

the acoustic data is in a data file [at column 55, lines 17-19, as the acoustic data is stored 
once and utilized later]; 

identifies words within that phoneme data [at column 54, lines 21-24, as obtain the word 
using DP in a case of detecting phonemes]; 

generates annotation data [at column 73, lines 1-36, as indicate the recognized data and 
link information to the original speech data]. 

38. Regarding claim 123, Ahmad and Miike make obvious the claimed limitations as a whole, 
because Ahmad [at columns 5-6] describes an annotation embodiment recognizable as a whole to 
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one versed in the art using speech received by any appropriate manner and transcribed by any 
conventional automatic speech recognizer by explicitly describing the content and functionality of 
the recited limitations as the following terminology: 

an input, an speech recognizer, and an annotation generator [at column 22, lines 52-57, as 
methods implemented on a computer using appropriate programs for manipulating]; 

converts a voice signal [at column 7, lines 54-57, as produce a text data set by analysis 
using any known speech recognition]; 

generates annotation data by combining phoneme data and word data [at column 12, 
lines 15-28, as construct a network top tier of words and next lower tier of phonemes]. 

However, Ahmad does not describe details of speech recognition. In particular, Ahmad 
does not describe phoneme recognition and phonemes representing the words. 

The discussion by Ahmad requires a method for automatic speech recognition, but merely 
any conventional method. Miike provides a conventional method. Accordingly, an artisan would 
be motivated to find known automatic speech recognition programs in order to implement 
Ahmad 's method of text/video/text time alignment. Ahmad " s [at abstract] application of search 
and access to databases would suggest finding speech recognition used in an similar application. 
Miike [at columns 52 ff], then, discloses speech recognition for data retrieval, and describes: 

receive an input voice signal [at column 55, lines 36-38, as input speeches of speakers]; 

convert it into phoneme data and words [at column 56, lines 15-24, as detect the phonemes 
and also detect the words of the utterance]; 

generate annotation data [at column 73, lines 1-36, as indicate the recognized data and link 
information to the original speech data]. 
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39. Regarding claim 125, Ahmad also describes: 

first, second, and third generators [at column 22, lines 52-57, as methods implemented on a 
computer using appropriate programs for manipulating]; 

generate data defining nodes within a lattice and links connecting the nodes [at column 13, 
lines 10-57, as construct a network/trellis representation including nodes and branches extending 
between nodes]; 

generate data associating the phonemes with the links [at column 15, lines 8-30, as form 
subnetworks of phonemes leading to a node (labeled with a word)]; 

generate data associating at least one word with a least one link [at column 14, lines 7-55, 
as represent various ways that words can be represented in subnetworks leading from an input 
node to an exit node]. 

40. Regarding claim 126, Ahmad also describes: 

generate the data defining the lattice in blocks of nodes [at column 18, lines 50-60, as 
subdivide the audio and text data used in the Viterbi method into parts]. 

41 . Regarding claim 127, Ahmad also describes: 

generate data defining time stamp information for each node [at column 18, lines 3-20, as 
each node contains a reference to the audio temporal data set from the network]. 



42. Regarding claim 128, Ahmad [at column 18, lines 65-66] also describes that the parts used 
for a divided Viterbi do not have to be equal in size, in general. Ahmad deemed it necessary to 
point out that the parts do not have to be of equal size; this suggests that Ahmad considered it 
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would have been obvious to one of ordinary skill in the art of subdividing text and audio data sets 
at the time of invention to make the division in Viterbi blocks of equal time duration. 

43. Regarding claim 129, Miike also describes: 

generate data which defines each block's location within a database [at column 75, 
lines 23-28, as store using information indicating storing position and the address in the storage 
unit]. 

44. Regarding claim 130, Ahmad also describes: 

the data file includes a time sequential signal [at column 18, lines 16-17, as the audio 
temporal data set]; 

generate time stamp data which is time synchronized with that signal [at column 18, 
lines 3-20, as each node contains a reference to the audio temporal data set]. 

45. Regarding claim 134, Ahmad also describes: 

define at least one node which is connected to a plurality of other nodes by a plurality of 
links [see Fig. 7B, item 701, Fig. 7C-1, Fig. 7C-2 and their descriptions especially at column 19, 
lines 10-23 of specifying the beginning of the paths through the transcription network followed by 
possible paths from one node to the next node being multiple word and phoneme branches]. 



46. Claims 141, 144, 145, 146, 147, and 151 set forth a method with limitations comprising the 
functionality associated with using the system recited in claims 122, 125, 127, 129, 130, and 134, 
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respectively. Because Ahmad and Miike describe and make obvious the similar limitations as 
indicated there, these claims thus are unpatentable accordingly. 

47. Regarding claim 158, Ahmad also describes: 

generate the data defining the lattice in blocks [at column 18, lines 50-60, as subdivide the 
audio and text data into parts]; 

the blocks are arranged in a time-ordered sequence [see Fig. 4, items 434, 432a, 432b, 
432c, 432d, 432e, and their descriptions at columns 19-20 of the network 434 correspondence to 
regions of the text file 432 with time synchronicity in sequence a, b, c, d, e]. 

48. Regarding claim 159, Ahmad also describes: 

the lattice is associated with a time sequential signal [at column 18, lines 3-20, as each 
node contains a reference that can be traced temporally to the text data set and to the audio 
temporal data set]; 

the time-ordered blocks are synchronized with the time sequential signal [see Fig. 4, items 
434, 432a, 432b, 432c, 432d, 432e, and their descriptions at columns 19-20 of portions of the 
network 434 correspondence to portions of the text file 432a-432e with time synchronicity]. 

49. Regarding claim 162, Ahmad also describes: 

a phoneme lattice and a number of different possible phoneme strings that correspond to 
speech [see Fig. 7B, Fig. 7C-1, Fig. 7C-2 and their descriptions especially at column 19, lines 10- 
23, as the paths through the transcription network of multiple possible branches that are associated 
with phonemes to provide possible paths between nodes]; 





y 
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combine the phonemes with the identified words, [at column 12, lines 15-28, as construct 
a network top tier of words and next lower tier of phonemes]. 

the phonemes are combined as a lattice [at column 15, lines 15-24, as the phoneme 
subnetworks into which a word may be composed]. 

50. Claim 163 sets forth additional limitations similar to limitations set forth in claim 162. 
Ahmad and Miike describe and make obvious the limitations as indicated there. 

51. Regarding claim 164, Miike also describes: 

generate header data relating to the recognizer that generated the data for the annotation [at 
column 69, lines 1-27, as auxiliary information may be stored by being attached to a header 
portion of the stored recognition result]. 

However, Miike is here describing the recognition result of document image processing, 
and does not explicitly describe header data for the speech recognition embodiment. With this 
suggestion by Miike to store the auxiliary information about the recognition in a header because 
that make the retrieval faster, it would have been obvious to one of ordinary skill in the art of file 
storage at the time of invention to use this same concept of header storage for auxiliary 
information relating to the phoneme recognizer to make their retrieval faster. 



52. Claim 166 sets forth additional limitations similar to limitations set forth in claim 164. 
Ahmad and Miike describe and make obvious the additional limitations as indicated there. 
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Ahmad and Miike and Bird et al 

53. Claims 160-161 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ahmad et 
al. [US Patent 6,172,675] in view of Miike [US Patent 5,787,414] and Steven Bird and Mark 
Liberman, "Towards a Formal Framework for Linguistic Annotations," fBird et al.) , already of 
record. 

54. Regarding claim 160, Ahmad also describes: 

generate node data defining nodes within the lattice [at column 13, lines 10-57, as 
construct a network/trellis representation including nodes]; 

each node represents a point in time within the associated time sequential signal [at 
column 18, lines 3-20, as each node contains a reference to the audio temporal data set from the 
network]. 

a word or phoneme begins or ends at that node and time [see Fig. 7B, item 701, Fig. 7C-1, 
Fig. 7C-2 and their descriptions especially at column 14-15 of the multiple word and phoneme 
branches beginning at the node that ends the preceding word and ending at the node that begins 
the next word on a path through the transcription network]. 

However, Ahmad does not explicitly describe that the node's time is generated relative to 
the time index of the node's block in the lattice. 

Bird et al. [at section 3.1 and Fig. 2] describes an annotation index of a data file in the form 
of a phoneme and word lattice. Bird et al. also describes: 

the data is arranged in blocks [at section 3.1, as files containing sets of lines 2360 through 
12257, etc. and sets of lines 2360 through 49066, etc.]; 
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the blocks are a time-ordered sequence [at section 3.2, as 2360 and 49066 information 
about the time and, for example, 12257 time between them]; 

each lattice block includes an associated time index [at section 3.2, as 2360 and 49066 
information about the time and, for example, 12257 time between them]; 

the node includes a time value [at section 3. 1, as lines showing times 0, 2360, 3720, 5200, 

etc.]; 

the time value is offset [at section 4.8, as time is offset]; 

the offset defines the node time relative to the time index of the block [at section 5.2, as 
annotation vertices temporally constrained to an assigned temporal locus of a semi-anchored 
vertex with reference to section 3]. 

Bird et al. 's [at section 3.1 and Fig. 2] embodiment describes a network phonetic index 
with similarities to the one of Ahmad in correspondence to an audio corpus . In view of the 
similarities between Bird et al. 's and Ahmad 's lattice representation , an artisan would have found 
it obvious at the time of invention to use Bird et al. 's concept of time offset within a division of 
the entire lattice relative to the division's own time in the overall sequence because that would 
relieve the burden when processing paths only within the particular portion of the lattice that does 
not need reference to other parts of the lattice, for example, Ahmad 's separate news stories. 



55. Claim 161 sets forth additional limitations similar to limitations set forth in claim 160. 
Ahmad . Miike. and Bird et al. describe and make obvious the additional limitations as indicated 
there. 
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Allowable Subject Matter 

56. Claims 165, 167, and 169 recite subject matter that is not taught or made obvious by prior 
art when considered with the limitations of the base claim and intervening claims. 

a. Claim 165 requires that a header storage area identify the language and the 
phoneme sets that were used by the ASR system, and that ASR system generated the annotation. 
The closest prior art (Mnke, Moshier ) suggests or describes ASR generating annotations and ASR 
information stored in a header. Neither describes header storage or language indications and of 
word or phoneme sets identities. 

b. Claim 167, and by dependency claim 169, at least set forth allowable material 
similar to claim 164. 

Response to Arguments 

57. The prior Office action, mailed August 14, 2003 (paper 14), objects to the drawings, title, 
specification, and claims, and rejects claims under 35 USC § 101 and § 103, citing Bird et al. and 
others. The Applicant's arguments and changes in PRELIMINARY AMENDMENT AND 
PETITION FOR EXTENSION OF TIME filed January 14, 2004 (paper 16) have been fully 
considered with the following results. 

58. With respect to objection to the drawings, the changes entered by amendment are assisted 
by the drawings in communicating the invention. Accordingly, the objection is removed. 

59. With respect to objection to the title, the changes previously entered by amendment are 
sufficiently descriptive. Accordingly, the objection is removed. 
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60. With respect to objection to the specification as lacking proper antecedence for claimed 
matter, the claimed subject matter remains. Accordingly, the objection is maintained. 

61. With respect to objections to the claims needing clarification, the objections no longer 
apply because those claims have been canceled. Please see new grounds of objection. 

62. With respect to rejections of claims under 35 USC § 101, the rejections no longer apply 
because the rejected claims have been canceled. Please see new grounds of rejection. 

63. With respect to rejections of claims under 35 USC § 103 , the rejections no longer apply 
because the rejected claims have been canceled. 

64. The Applicant's submission includes comments that appear relevant to prior art used in an 
earlier Office action as grounds of those rejection of claims that were similar to the currently 
pending claims: 

a. The Applicant appears to remark that it would not have been obvious to generate 
annotations for the TIMIT, PARTITUR, and EMU data bases by automatic recognition 
apparatuses for various reasons relating to these databases. The Examiner comments that the 
features upon which the Applicant's comment relies are not recited in the rejected claims. 

b. The Applicant appears to remark that Kupiec 's index does not meet the limitation 
of the claimed annotation data because Kupiec 's index does not annotate Kupiec 's text database. 
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The Examiner comments that the features upon which the Applicant's remark relies are not recited 
in the claims anticipated by Kupiec . 

c. The Applicant appears to remark that Kupiec does not combine the phoneme data 
converted from voice and the words converted from voice to generate data. The Examiner agrees, 
because Kupiec 's combined phonetic information and words were generated from a phonetic 
dictionary file and reference encyclopedia file. 

d. The Applicant appears to remark that Bird et al. 's phonetic data network and 
Kupiec 's network embodiment of the phonetic index have completely different structures. The 
Examiner comments that at least both have a network of phonetic information. Also, the network 
of each is used for pointing purposes. 

e. The Applicant appears to remark that the presence of software does not necessarily 
indicate automatic generation of the data of Bird et al. 's network. The Examiner comments that 
Bird et al. does not explicitly describe the software's operation. 



Conclusion 

65. Any response to this action should be mailed to: 

Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 223 13-1450 

or faxed to: 

(703) 872-9306, (for formal communications intended for entry) 

Or: 

(703) 872-9306, (for informal or draft communications, and please label 
"PROPOSED" or "DRAFT") 
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Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington, VA (Sixth Floor, Receptionist) 

66. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Donald L. Storm, of Art Unit 2654, whose telephone number is 
(703) 305-3941. The examiner can normally be reached on weekdays between 8:00 AM and 4:30 
PM Eastern Time. If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Richemond Dorvil can be reached on (703) 305-9645. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Donald L. Storm 
Patent Examiner 
Art Unit 2654 
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